Uncertainty visualization of gaze estimation to support operator-controlled calibration

      In this paper, we investigate how visualization assets can support the qualitative evaluation of gaze estimation uncertainty. Although eye tracking data are commonly available, little has been done to visually investigate the uncertainty of recorded gaze information. This paper tries to fill this gap by using innovative uncertainty computation and visualization. Given a gaze processing pipeline, we estimate the location of this gaze position in the world camera. To do so we developed our own gaze data processing which give us access to every stage of the data transformation and thus the uncertainty computation. To validate our gaze estimation pipeline, we designed an experiment with 12 participants and showed that the correction methods we proposed reduced the Mean Angular Error by about 1.32 cm, aggregating all 12 participants’ results. The Mean Angular Error is 0.25° (SD=0.15°) after correction of the estimated gaze. Next, to support the qualitative assessment of this data, we provide a map which codes the actual uncertainty in the user point of view.

EyeFlow: pursuit interactions using an unmodified camera

We investigate the smooth pursuit eye movement based interaction using an unmodified off-the-shelf RGB camera. In each pair of sequential video frames, we compute the indicative direction of the eye movement by analyzing flow vectors obtained using the Lucas-Kanade optical flow algorithm. We discuss how carefully selected low vectors could replace the traditional pupil centers detection in smooth pursuit interaction. We examine implications of unused features in the eye camera imaging frame as potential elements for detecting gaze gestures. This simple approach is easy to implement and abstains from many of the complexities of pupil based approaches. In particular, EyeFlow does not call for either a 3D pupil model or 2D pupil detection to track the pupil center location. We compare this method to state-of-the-art approaches and ind that this can enable pursuit interactions with standard cameras. Results from the evaluation with 12 users data yield an accuracy that compares to previous studies. In addition, the benefit of this work is that the approach does not necessitate highly matured computer vision algorithms and expensive IR-pass cameras

Improving eye-tracking calibration accuracy using symbolic regression

Eye tracking systems have recently experienced a diversity of novel calibration procedures, including smooth pursuit and vestibulo-ocular reflex based calibrations. These approaches allowed collecting more data compared to the standard 9-point calibration. However, the computation of the mapping function which provides planar gaze positions from pupil features given as input is mostly based on polynomial regressions, and little work has investigated alternative approaches. This paper fills this gap by providing a new calibration computation method based on symbolic regression. Instead of making prior assumptions on the polynomial transfer function between input and output records, symbolic regression seeks an optimal model among different types of functions and their combinations. This approach offers an interesting perspective in terms of flexibility and accuracy. Therefore, we designed two experiments in which we collected ground truth data to compare vestibulo-ocular and smooth pursuit calibrations based on symbolic regression, both using a marker or a finger as a target, resulting in four different calibrations. As a result, we improved calibration accuracy by more than 30%, with reasonable extra computation time

Evaluating Stress Through Thermal Imaging Cameras

There is evidence that a high cognitive workload may hamper human performance. In the context of piloting, it is important to investigate the factors that induce such situations. This can be done using sensors that capture behavioral and physiological responses. However, used sensors are often cumbersome and require physical contact (EEC, fNIRS). We need sensors that can operate and access physiological parameters uninvasively. In this sense, thermal imagingbased physiological monitoring is an active research topic. It grants remote and unobtrusive access to the distribution of the temperature on the surface of objects and skin, by interpreting the emitted electromagnetic radiations reflected on the surface. Therefore, physiological behaviors can be remotely evaluated through thermal cameras. Recent developments of small and pervasive devices provide affordable mobile thermal image cameras

Utilisation du regard, analyse et interaction

Les technologies de suivi du regard permettent d’estimer les endroits dans une scène où un utilisateur regarde. Les capacités des dispositifs informatiques sont en perpétuelle croissance, ainsi que le besoin pour de nouveaux outils qui peuvent permettre de tirer profit de nouveaux moyens de communiquer (par exemple avec les yeux). Ces systèmes sont utilisés dans divers domaines, notamment dans l’interaction homme-machine, les études sur l’ergonomie et l’utilisabilité, et le transfert de connaissance. Par exemple, pour les personnes ayant une déficience motrice ou un handicap physique, la saisie de texte par le regard peut être un important moyen d’interaction. En outre, ces outils peuvent être également utilisés pour comprendre les comportements visuels d’un pilote cherchant de l’information dans un cockpit.Toutefois, un certain nombre de barrières existent et rendent l'utilisation de ces outils moins précise et donc moins efficace. L’un des problèmes majeurs est le décalage entre la position réelle et la position estimée du point de regard de l’utilisateur. Celui-ci, sujet à des erreurs de précision, ne permet pas encore de se servir pleinement du canal d’interaction que peut offrir le regard. Suite aux progrès récents des technologies de détection de mouvements oculaires, on s’intéresse de plus en plus aux systèmes abordables, pas coûteux, précis et facilement utilisables.Cette thèse porte sur différents aspects de la recherche sur les mouvements oculaires. Elle propose de nouvelles approches qui permettent d’augmenter la précision de calcul de l’estimation du regard. En effet, nous montrerons qu’il existe des fonctions polynomiales plus adaptées que les modèles utilisés dans la littérature, et même dans les outils commercialisés. Nous proposons donc une nouvelle façon de construire automatiquement de nouveaux modèles qui sont mieux adaptés pour l’estimation du regard, et ce, pour tous les types de méthodes de calibration basées sur les fonctions polynomiales, connus à ce jour. Nous présentons aussi de nouvelles stratégies pour l’interaction basée sur le regard, ne nécessitant aucune calibration préalable. En plus de proposer des stratégies pour améliorer la précision et des techniques d’interaction, des outils de visualisation et d’exploration sont également proposés. Dans cette thèse, nous contribuons alors sous quatre différents angles: les techniques de calibration, l’interaction basée sur le regard, la visualisation et l’exploration.Eye-tracking technologies have a long history in various domains as a tool for understanding human visual behavior. They estimate the locations in a scene where a user is looking. These systems are used in various domains including human-computer interaction, usability studies, and learning transfer. As an example, gaze-based text entry allows interacting with computing systems for people with motor impairments, physical disabilities or lower muscle controls.They are also used to comprehend the visual behaviors of a pilot searching for information in a cockpit. The capabilities of computing devices are growing, so as the demand for new tools that take advantage of other sense organs (e.g., the eyes). However, a number of barriers still exist and make these devices less accurate and difficult to use in daily activities. One of these problems is the shift between the actual and the estimated position of the user’s point-of-regard,which systematically comes from the eye-tracking systems’ accuracy and precision. Following recent advances, there is an increasing interest in affordable systems that have the potential to be more accurate and, researchers are continually investigating novel approaches. This thesis covers different issues of eye movement research. It proposes the use of novel approaches as a step towards overcoming these accuracy and precision issues. More specifically, we introduce novel strategies for detecting mapping functions for gaze estimation and calibration-free gaze interaction. In addition to proposing frameworks and strategies for improving accuracy, new calibration procedures and patterns are also revealed and discussed. In this dissertation, we address these issues in three different ways: calibration and mapping functions, calibration-less gaze interaction, visualization and exploration. We present four main contributions. First, we present a new method for calibrating state-of-the-art eye trackers with better accuracy. Second, we present a new gaze-based authentication method which works without any prior calibration, and can be extended to any alphanumeric-based input modality. Third, we present an uncertainty visualization approach. Finally, a method of analyzing eye movements data and aircraft trajectories using a novel brushing technique is proposed

Utilisation du regard, analyse et interaction

Eye-tracking tools estimate the locations in a scene where a user is fixating on. They are used in various domains including human-computer interaction (HCI) and learning transfer. As an example, gaze-based text entry allows interacting with computing systems remotely without touching the interface. They are also used to comprehend the visual behaviors of a pilot searching for information in a cockpit. However, a number of barriers still exists and makes these devices less accurate and difficult to use in daily activities. One of these problems is the shift between the actual and the estimated position of the user’s point-of-regard, which systematically comes from the eye-tracking systems’ accuracy. Following recent advances, there is an increasing interest in affordable systems that have the potential to be more accurate and, researchers are continually investigating novel approaches.This thesis covers different issues of eye movement research. It proposes the use of novel approaches as a step towards overcoming these accuracy issues. More specifically, we introduce novel strategies for detecting mapping functions for gaze estimation and calibration-free gaze interaction. In addition to proposing frameworks and strategies for improving accuracy, new calibration procedures and patterns are also revealed and discussed. In this thesis, we address these issues in three different ways: calibration and mapping functions, Human-computer Interaction using the eyes, visualization and exploration. We present four main contributions. First, we present a new method for calibrating state-of-the-art eye trackers with better accuracy. Second, we present a new gaze-based authentication method which works without any prior calibration, and can be extended to any alphanumeric-based input modality. Third, we present an uncertainty visualization approach. Finally, a method of analyzing eyemovements data and aircraft trajectories using a novel brushing technique is proposed.Les technologies de suivi du regard permettent d’estimer les positions du regard d'un utilisateur dans une scène. Les capacités des dispositifs informatiques sont en perpétuelle croissance, ainsi que le besoin pour de nouvelles formes d'analyse ou d'interaction. Ces outils sont utilisés dans divers domaines, notamment dans l’interaction homme-machine, les études sur l’ergonomie et l’utilisabilité, et le transfert de connaissance. Par exemple, pour les personnes ayant un handicap moteur, la saisie de texte par le regard peut être un moyen d’interaction. En outre, ces outils peuvent être également utilisés pour comprendre les comportements visuels d’un pilote cherchant de l’information dans un cockpit. Toutefois, un certain nombre de barrières continuent d'exister et rendent ces outils moins précis et donc moins efficaces. L’un des problèmes majeurs est le décalage entre la position réelle et la position estimée du point de regard de l’utilisateur. Celui-ci, sujet à des erreurs de précision, ne permet pas encore de se servir pleinement du canal d’interaction que peut offrir le regard. À la suite des progrès récents, on s’intéresse de plus en plus aux systèmes abordables, pascoûteux, précis et facilement utilisable. Cette thèse porte sur différents aspects de la recherche sur les mouvements oculaires. Elle propose de nouvelles approches qui permettent d’augmenter la précision de calcul de l’estimation du regard. Nous proposons une nouvelle façon de construire automatiquement de nouveaux modèles qui sont mieux adaptés pour l’estimation du regard, et ce, pour tous les types de méthodes de calibration connus à ce jour. Nous présentons aussi de nouvelles stratégies pour l’interaction basée sur le regard, ne nécessitant aucune calibration préalable. En plus de proposer des stratégies pour améliorer la précision et des techniques d’interaction, des outils de visualisation et d’exploration sont également proposés. Nous contribuons alors sous quatre différents angles : les techniques de calibration, l’interaction homme-machine basée sur le regard, la visualisation et l’exploration

Eye Gesture in a Mixed Reality Environment

International audienceUsing a simple approach, we demonstrate that eye gestures could provide a highly accurate interaction modality in a mixed reality environment. Such interaction has been proposed for desktop and mobile devices. Recently, Gaze gesture has gained a special interest in Human-Computer Interaction and granted new interaction possibilities, particularly for accessibility. We introduce a new approach to investigate how gaze tracking technologies could help people with ALS or other motor impairments to interact with computing devices. In this paper, we propose a touch-free, eye movement based entry mechanism for mixed reality environments that can be used without any prior calibration. We evaluate the usability of the system with 7 participants, describe the implementation of the method and discuss its advantages over traditional input modalities

Improving eye-tracking calibration accuracy using symbolic regression.

Eye tracking systems have recently experienced a diversity of novel calibration procedures, including smooth pursuit and vestibulo-ocular reflex based calibrations. These approaches allowed collecting more data compared to the standard 9-point calibration. However, the computation of the mapping function which provides planar gaze positions from pupil features given as input is mostly based on polynomial regressions, and little work has investigated alternative approaches. This paper fills this gap by providing a new calibration computation method based on symbolic regression. Instead of making prior assumptions on the polynomial transfer function between input and output records, symbolic regression seeks an optimal model among different types of functions and their combinations. This approach offers an interesting perspective in terms of flexibility and accuracy. Therefore, we designed two experiments in which we collected ground truth data to compare vestibulo-ocular and smooth pursuit calibrations based on symbolic regression, both using a marker or a finger as a target, resulting in four different calibrations. As a result, we improved calibration accuracy by more than 30%, with reasonable extra computation time

Interactive Trajectory Modification and Generation with FPCA

Best student paper award ICRAT 2020International audienceMoving object analysis is a constantly growing field with numerous concrete applications in terms of traffic understanding, prediction and simulation. While many algorithms and analytic processes exist, there are still areas of investigation with novel trajectory analysis methods. As such, the geometric information analyses data with respect to its statistical distribution along extracted dimensions. This opens new ways of gaining a better understanding of large and complex trajectory data sets while providing flexible data manipulations. In this paper, we report our investigations with the development of an interactive methodology based on the geometric information analytic process where users can analyze trajectories sets, cluster and deform them maintaining the actual statistical properties of the investigated trajectories. As a contribution, this paper shows how geometric information can provide novel support for trajectory analyses taking into account the statistical properties of the investigated clusters. We also provide recommendations of good usage of such techniques with actual examples validated by a a domain expert of air traffic flow analysi

Thermal Imaging of the Face: Mental Workload Detection in Flight Simulator

Thermography-based physiological measurement is a contact-free approach that can be particularly helpful for detecting pilots’ mental state in operational settings. In particular, thermal infrared imaging of the face is a powerful, unobtrusive and non-invasive tool that enables rapid and automatic analysis of changes in regional facial blood flow. These blood flow changes index sympathetic activity and are measured by capturing thermal imprints of particular facial regions such as nose or forehead. Although several studies suggest a relationship between mental workload and facial thermoregulation profile, evidence about this link has not yet been sufficiently investigated. In this work, we investigated how thermal measures can allow robust and continuous assessment of mental workload variations of pilots undergoing simulated flight tasks. We analyzed thermal data and heart rate of 20 participants in a flight simulator. Mental workload was modulated by the difficulty of the landing scenario or by an in-flight N-back task. Participants also performed a resting task (called cool-off) in the flight simulator. Thermal imprints did not vary significantly with landing difficulty or N-back difficulty. However, we found that the nose tip and nose area became significantly colder (signal slope was negative) during all piloting scenarios vs the rest period. Heat rate was slightly more sensitive to the piloting difficulty since it was marginally higher during the difficult vs easy landing. Results are promising but further analysis is needed to confirm that the thermal measures could identify fine-grained mental workload variations in a flight simulator setting